4 research outputs found
MeciFace: Mechanomyography and Inertial Fusion based Glasses for Edge Real-Time Recognition of Facial and Eating Activities
The increasing prevalence of stress-related eating behaviors and their impact
on overall health highlights the importance of effective monitoring systems. In
this paper, we present MeciFace, an innovative wearable technology designed to
monitor facial expressions and eating activities in real-time on-the-edge
(RTE). MeciFace aims to provide a low-power, privacy-conscious, and highly
accurate tool for promoting healthy eating behaviors and stress management. We
employ lightweight convolutional neural networks as backbone models for facial
expression and eating monitoring scenarios. The MeciFace system ensures
efficient data processing with a tiny memory footprint, ranging from 11KB to
19KB. During RTE evaluation, the system achieves impressive performance,
yielding an F1-score of < 86% for facial expression recognition and 90% for
eating/drinking monitoring, even for the RTE of an unseen user.Comment: Submitted to Nature Scientific Report
InMyFace: Inertial and Mechanomyography-Based Sensor Fusion for Wearable Facial Activity Recognition
Recognizing facial activity is a well-understood (but non-trivial) computer
vision problem. However, reliable solutions require a camera with a good view
of the face, which is often unavailable in wearable settings. Furthermore, in
wearable applications, where systems accompany users throughout their daily
activities, a permanently running camera can be problematic for privacy (and
legal) reasons. This work presents an alternative solution based on the fusion
of wearable inertial sensors, planar pressure sensors, and acoustic
mechanomyography (muscle sounds). The sensors were placed unobtrusively in a
sports cap to monitor facial muscle activities related to facial expressions.
We present our integrated wearable sensor system, describe data fusion and
analysis methods, and evaluate the system in an experiment with thirteen
subjects from different cultural backgrounds (eight countries) and both sexes
(six women and seven men). In a one-model-per-user scheme and using a late
fusion approach, the system yielded an average F1 score of 85.00% for the case
where all sensing modalities are combined. With a cross-user validation and a
one-model-for-all-user scheme, an F1 score of 79.00% was obtained for thirteen
participants (six females and seven males). Moreover, in a hybrid fusion
(cross-user) approach and six classes, an average F1 score of 82.00% was
obtained for eight users. The results are competitive with state-of-the-art
non-camera-based solutions for a cross-user study. In addition, our unique set
of participants demonstrates the inclusiveness and generalizability of the
approach.Comment: Submitted to Information Fusion, Elsevie
Remaining Useful Life Prediction of Lithium-ion Batteries using Spatio-temporal Multimodal Attention Networks
Lithium-ion batteries are widely used in various applications, including
electric vehicles and renewable energy storage. The prediction of the remaining
useful life (RUL) of batteries is crucial for ensuring reliable and efficient
operation, as well as reducing maintenance costs. However, determining the life
cycle of batteries in real-world scenarios is challenging, and existing methods
have limitations in predicting the number of cycles iteratively. In addition,
existing works often oversimplify the datasets, neglecting important features
of the batteries such as temperature, internal resistance, and material type.
To address these limitations, this paper proposes a two-stage remaining useful
life prediction scheme for Lithium-ion batteries using a spatio-temporal
multimodal attention network (ST-MAN). The proposed model is designed to
iteratively predict the number of cycles required for the battery to reach the
end of its useful life, based on available data. The proposed ST-MAN is to
capture the complex spatio-temporal dependencies in the battery data, including
the features that are often neglected in existing works. Experimental results
demonstrate that the proposed ST-MAN model outperforms existing CNN and
LSTM-based methods, achieving state-of-the-art performance in predicting the
remaining useful life of Li-ion batteries. The proposed method has the
potential to improve the reliability and efficiency of battery operations and
is applicable in various industries, including automotive and renewable energy
Facial Muscle Activity Recognition with Reconfigurable Differential Stethoscope-Microphones
Many human activities and states are related to the facial muscles’ actions: from the expression of emotions, stress, and non-verbal communication through health-related actions, such as coughing and sneezing to nutrition and drinking. In this work, we describe, in detail, the design and evaluation of a wearable system for facial muscle activity monitoring based on a re-configurable differential array of stethoscope-microphones. In our system, six stethoscopes are placed at locations that could easily be integrated into the frame of smart glasses. The paper describes the detailed hardware design and selection and adaptation of appropriate signal processing and machine learning methods. For the evaluation, we asked eight participants to imitate a set of facial actions, such as expressions of happiness, anger, surprise, sadness, upset, and disgust, and gestures, like kissing, winkling, sticking the tongue out, and taking a pill. An evaluation of a complete data set of 2640 events with 66% training and a 33% testing rate has been performed. Although we encountered high variability of the volunteers’ expressions, our approach shows a recall = 55%, precision = 56%, and f1-score of 54% for the user-independent scenario(9% chance-level). On a user-dependent basis, our worst result has an f1-score = 60% and best result with f1-score = 89%. Having a recall ≥60% for expressions like happiness, anger, kissing, sticking the tongue out, and neutral(Null-class)